1. Field of Technology
The present invention relates to a device and method for processing an image acquired by digital signal processing of an image signal output from a MOS (metal oxide semiconductor) imaging sensor for recording a high resolution still JPEG image, or a Motion JPEG, JPEG 2000, MPEG-1, MPEG-2, or MPEG-4, or H.263 or H.264 encoded video signal. More particularly, the invention relates to a digital camera, hand-held camera, or other mobile imaging device.
2. Description of Related Art
NMOS and CMOS image sensors are developing rapidly as the next-generation successor to CCD image sensors for applications in both video and still portable digital cameras, hand-held cameras, and cell phone cameras. One advantage of MOS sensors compared with CCD sensors is that MOS sensors do not have the signal transfer portion of a CCD device, and require only drive and sensing lines. MOS sensors will be expected to afford a larger photodiode aperture and hence a wider dynamic range. This gives MOS sensors a clear advantage as resolution increases. CCD sensors also require multiple power supply lines and wiring in order to handle signal transfer, while MOS sensors can use a single power supply line. MOS sensors offer the advantages of an extremely simple power supply and wiring design, and the incumbent lower cost.
Image signal processing in digital cameras, hand-held cameras, and cell phone cameras first writes the input signals from the CCD or MOS image sensor line by line to relatively inexpensive SDRAM (synchronous DRAM) for temporary storage. The input signals are written one line at a time pixel by pixel starting from the first sensor in line one. After the line one signals are written to memory, line 2 is written, then line 3, and so forth until all lines in one frame have been written to SDRAM.
The frame signal is then read from SDRAM for signal processing such as zooming in or out to enlarge or reduce the image. After this operation is completed, the signal is written to SDRAM again. The signal is then read for image signal compression and converted to a JPEG or other image compression format suitable for recording. The compressed image signal is then written again to SDRAM, and finally read from SDRAM by high speed DMA (direct memory access) control, for example, and transferred to an external nonvolatile storage device.
Signal processing for capturing a single still image from a digital still camera is described above. Recent digital cameras also typically feature a continuous exposure mode enabling capturing multiple still images in rapid succession.
When a CCD sensor is used with a mechanical shutter, all pixels in the CCD sensor are charged simultaneously to the shutter starting to open, and charging all pixels stops simultaneously to the shutter closing. This ability to simultaneously charge all pixels is one measure of imaging sensor performance.
Even when a mechanical shutter is not available as a means of assuring simultaneous charging of all pixels, if the charge is transferred to all pixels of the CCD sensor after exposure, the charge will not change even if the next exposure comes, and all charges detected by the CCD can be stored simultaneously at a known timing signal, thus assuring the synchronous operation of the CCD sensor.
MOS sensors, on the other hand, do not have the temporary charge storage capacity of a CCD sensor, and sensor charges are read sequentially pixel by pixel. As a result, there is a time lag between reading the first pixel and reading the last pixel. This is not a problem when capturing still images, but if there is movement in the subject image area, this time lag results in image distortion.
Various methods of solving this problem have been proposed. The simplest method is to provide a mechanical shutter to assure simultaneous exposure of all pixels when capturing still images. In this case the pixels are exposed simultaneously but read sequentially.
While a mechanical focal plane shutter that travels at the same speed as the exposure is used to produce a line by line exposure when the overall scene is dark, this focal plane shutter cannot provide sufficient exposure when the scene is bright. The overall scene is therefore exposed using a mechanical shutter and an on/off electronic shutter signal is generated to set the reset timing for each line, thereby achieving an electronic shutter with a wide dynamic range enabling exposure in low light situations. See Japanese Unexamined Patent Appl. Pub. 2003-32556.
The foregoing method is described further below.
FIG. 13 is a function block diagram of a camera enabling simultaneous exposure by a MOS sensor, and FIG. 14 is a flow chart showing the operation of this device. A conventional device assuring the simultaneous exposure by this MOS sensor is described first with reference to FIG. 13.
As shown in FIG. 13, this conventional digital camera has an imaging lens 50, an imaging element 51 for converting the optical signal from the imaging lens 50 to an electric signal, a signal processing unit 52, memory devices 53 and 54, a lens aperture drive unit 56 for driving the aperture of the imaging lens 50, a drive unit 55 for driving the imaging element, a lens information detector 58 for detecting such lens information as the position of the imaging lens, a central control unit 57, an operating unit 59 used by the operator, and a light color detector 60.
Operation of this camera is described next.
The imaging element 51 photoelectrically converts the optical signal from the imaging lens 50 to an electric signal. The converted signal is written by the signal processing unit 52 to memory devices 53 and 54. The lens aperture drive unit 56 controls the aperture of the imaging lens 50. The drive unit 55 that drives the imaging element 51 is controlled by the central control unit 57, and the delay between when the first drive signal is applied to when the second drive signal is applied differs between pixels. The timing of applying the first drive signal and the timing of applying the second drive signal varies pixel by pixel.
An algorithm for preventing subject distortion when the flash is not used, and for image stabilization to improve blurring due to hand shaking is shown in FIG. 14.
The lens to subject distance is measured first (#405). The exposure time is then determined by the subject brightness measurement step (#410). The part of the image area to be captured is then determined by the main subject detection step (#415). The exposure area is then determined by the exposure area selection step (#420). Photoelectric conversion then starts in the photoelectric conversion starting step (S1) (#425). Operation then waits for photoelectric conversion to end in the photoelectric conversion step (#430). When photoelectric conversion ends, the decoded signal is output in the start of decoded signal output step (S2) (#435). This process extracts the subject part of the image area and minimizes distortion in a moving subject.